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I. INTRODUCTION 



ABSTRACT 

A new framework of compressive sensing (CS), namely statistical compres- 
sive sensing (SCS), that aims at efficiently sampling a collection of signals 
that follow a statistical distribution and achieving accurate reconstruction 
on average, is introduced. For signals following a Gaussian distribution, 
with Gaussian or Bernoulli sensing matrices of &(k) measurements, 
considerably smaller than the &(k\og(N/k)) required by conventional CS, 
where N is the signal dimension, and with an optimal decoder implemented 
with linear filtering, significantly faster than the pursuit decoders applied 
conventional CS, the error of SCS is shown tightly upper bounded by 
a p onstant times the &-best term approximation error, with overwhelming 

^p pbability The failure probability is also significantly smaller than that 
conventional CS. Stronger yet simpler results further show that for 
fin y sensing matrix, the error of Gaussian SCS is upper bounded by a 
^nstant times the &-best term approximation with probability one, and the 

^^und constant can be efficiently calculated. For signals following Gaussian 
mixture models, SCS with a piecewise linear decoder is introduced and 

Q5own to produce for real images better results than conventional CS based 
sparse models. 

i— Index Terms — Compressive sensing, Gaussian mixture models 

> 
U 

^Compressive sensing (CS) attempts to achieve accurate signal recon- 
duction while sampling signals at a low sampling rate, typically far 
•-smaller than the Nyquist/Shannon rate. Let x £ R N be a signal of interest, 
M. MxN a non _ a d a pff ve sensing matrix (encoder), consisting of M<C N 
►measurements, y = <£x £ R M a measured signal, and A a decoder used to 
Jmpon struct x from <£x. CS develops encoder-decoder pairs A) such that 

a small reconstruction error x — A(<£x) can be achieved. 
^^Reconstructing x from <£x is an ill-posed problem whose solution 
requires some prior information (model) on the signal. Instead of the 
Ji&quency band-limit signal model assumed in classic Shannon sampling 
Sne ory, conventional CS adopts a sparse signal model (manifold models 
1_ ave been considered as well 0), (8)), i.e., there exists a dictionary, 
pically an orthogonal basis ¥ £ M. NxN , sl linear combination of whose 
columns generates an accurate approximation of the signal, x « ^a, with 
fee coefficients a[m], 1 < m < N, having their amplitude decaying fast 
Imager being sorted. For signals following the sparse model, it has been 
^hjown that using some random sensing matrices such as Gaussian and 
^jrnoulli matrices with M = @(k\og(N /k)) measurements, and an l\ 
minimization or a greedy matching pursuit decoder A promoting sparsity, 
with high probability CS leads to accurate signal reconstruction. The 
obtained approximation error is tightly upper bounded by a constant times 
the &-best term approximation error, the minimum error that one may 
achieve by keeping the k largest coefficients in a (6), (9), ifTOl . 

The present paper introduces a novel framework of CS, namely statistical 
compressive sensing (SCS). As opposed to conventional CS that deals with 
one signal at a time, SCS aims at efficiently sampling a collection of signals 
and having accurate reconstruction on average. Instead of restricting to 
sparse models, SCS works with general Bayesian models. Assuming that 
the signals x follow a distribution with probability density function (pdf) 
p(x), SCS designs encoder-decoder pairs (<£,A) so that the average error 



£ x ||x-A(<£x) 



■/i 



x-A(®x)\\xp(x)dx, 



(1) 



where || • ||x is a norm, is small. As an important example, SCS with 
Gaussian models is here shown to have improved performance (bounds) 
relative to conventional CS, the signal construction calculated with an 
optimal decoder A implemented via a fast linear filtering. Moreover, for 



Gaussian mixture models (GMM), SCS with a piecewise linear decoder is 
introduced and shown to be very effective. 

The motivation of SCS with Gaussian models is twofold. First, con- 
trolling the average error over a collection of signals is useful in signal 
acquisition, not only because one is often interested in acquiring multiple 
signals in real applications, but also because more effective processing of 
an individual signal, an image or a sound for example, is usually achieved 
by dividing the signal in local subparts, patches or short-time windows for 
instance, and a signal can be regarded as a collection of subpart signals fTTl . 
1 18 1. In addition, Gaussian mixture models (GMM), which model signals or 
subpart signals with a collection of Gaussians, have been shown effective 
in describing real signals, leading to excellent results in image inverse 
problems 1 18 1 and missing data estimation fT3l . 

After reviewing the optimal decoder for Gaussian signals in Section |n| 
a quick numerical check of the good performance of Gaussian SCS is 
first given in Section [ill] Section |lV] analyzes this performance following 
a similar mathematical approach as the one adopted in conventional CS 
performance analysis |9|. This result shows that with the same random 
matrices as in conventional CS, but with a considerably reduced number 
M = &(k) of measurements, and with the optimal decoder implemented 
with linear filtering, significantly faster than the decoders applied in 
conventional CS, the average error of Gaussian SCS is tightly upper 
bounded by a constant times the &-best term approximation error with 
overwhelming probability, the failure probability being orders of magnitude 
smaller than that of conventional CS. Section |V| further shows stronger 
yet simpler results: For any sensing matrix, the average error of Gaussian 
SCS is upper bounded by a constant times the &-best term approximation 
with probability one, and the bound constant can be efficiently calculated. 
Section [Vl| introduces, for SCS with GMM, a piecewise decoding scheme 
based on an efficient maximum a posteriori expectation-maximization 
(MAP-EM) algorithm following [18), and shows better results on real 
images than those obtained with conventional CS. 

In the rest of this paper, we will assume without loss of generality Gaus- 
sian signal models x ~ ^(0,S), with mean zero and diagonal covariance 
matrix S = diag[X\ , . . . , Ajy], with X\ > A2 > . . . > the sorted eigenvalues. 
We can always center the data with respect to the Gaussian distribution 
and make a basis change with principal component analysis (PC A) (T8J. 
For Gaussian and Bernoulli matrices that are known to be universal, 
analyzing CS in the canonical basis or the PC A basis is equivalent El . 
The Gaussian distributions are assumed to be full rank, i.e., Xn > 0, since 
a degenerated Gaussian can be regarded as a full-rank Gaussian within a 
reduced dimension. 

II. OPTIMAL DECODER FOR GAUSSIAN SCS 

It is well-known that the optimal decoders for Gaussian signals are 
calculated with linear filtering: 

Theorem 1. [12] Let x £ R N be a random vector with prior pdf o/K(0,S), 
and £ R MxA ^, M <N, be a sensing matrix. From the measured signal 
y = <E>x £ R M , the optimal decoder A that minimizes the mean square 
error (MSE) E x [\\x - A(3>x)||^] = mm f E x [\\x - f(®x)\\j}, as well as the 
mean absolute error (MAE) E x [\\x — A(3>x)||i] = miny£' x [||x — /(ta)||i], 
where f is any mapping from R M — > R^, is obtained with a linear MAP 
estimator, 

A(<Dx) = argmax/?(x|y) = S<5> T (<5>S<Z> T y l (<S>x), (2) 

and the resulting error 77 = x — A(4>x) is Gaussian with mean zero and 
with covariance matrix = E x [r\i(] T } = S — S<£ r (<I>S<£ r ) -1 <£S, whose 
trace yields the MSE of SCS 

E x [\\x-A(®x)\\l] = 7>(S-S<£ r (<£S<£ r r 1 <£S). (3) 



In contrast to conventional CS, for which the l\ minimization or greedy 
matching pursuit decoders, calculated with iterative procedures, have been 
shown optimal under certain conditions on <I> and the signal sparsity |6|, 
1 10 1, Gaussian SCS enjoys the advantage of having an optimal decoder (5| 
calculated via a linear filtering for any <£. 

Corollary 1. If a random matrix <G R MxA ^ is drawn independently to 
sense each x, with all the other conditions as in Theorem [7] the MSE of 
SCS is 

^Hx-A^H^^IrKS-S^^S^)- 1 ^)]. (4) 

The MSE of Gaussian SCS has closed-forms ([3}, (4), however, a 
mathematical analysis of {3} and ([4} seems uneasy due to the complexity 
of the involved matrix expression. The next section evaluates these values 
through Monte Carlo simulations, preceding the theoretical bounds later 
developed. 

III. PERFORMANCE OF GAUSSIAN SCS 
- A NUMERICAL ANALYSIS AT FIRST 

This section numerically evaluates the MSE of Gaussian SCS, and com- 
pares it with the minimal MSE generated by the best &-term approximation. 
Under the Gaussian signal model x ~ c/K(0,S) with sorted eigenvalues 
Ai > A2 > . . . > A^, it is well-known that the best &-term approximation 
error, 

<J k ({x}) x =E x \\x-x K \\ x and o k ({x})l = E x \\x-x K \\l, (5) 
is obtained with a linear projection to the first k entries, i.e., x%[n] = x[n], 
\/n £ K = {1, . . . , k}, and x K [n] = otherwise 1 14]. Note that G k ({x})l = 
Hm=k+l^ m ' Th^ s b est &-term approximation sensing is impractical with 
GMM, since the Gaussian assignment of a signal is unknown at the 
acquisition moment, GMM describing real data much better than a single 
Gaussian model 1 18 1. 

A power decay of the eigenvalues [14], 

l m =m~ a , l<m<N, (6) 
N = 64, is assumed in the Monte Carlo simulation. An independent random 
Gaussian matrix realization is applied to sense each signal x |9 |, and (4| 
is used to calculate the MSE of SCS. 

Figures [T] (a) and (c)-top plot the MSE of SCS and that of the best k- 
term approximation, as well as their ratio as a function of k for Gaussian 
signals with a typical eigenvalue decay a = — 3. With k increasing, both 
MSEs decrease, their ratio being almost constant at about 3.7. The same 
is plotted in figures [T] (b) and (c)-bottom, with k fixed at a typical value of 
10, and a varying from 1 to 4. When a increases, the eigenvalues decay 
faster, and the MSEs for both methods decrease. The ratio increases almost 
linearly with a. 




Fig. 1. Comparison of the MSE of SCS and that of the best &-term approxi- 
mation for Gaussian signals. See text for details. 



These results indicate a good performance of Gaussian SCS, its MSE 
is only a small number of times larger than that of the best &-term 
approximation. The next sections provide mathematical analysis of this 
performance. 

IV. PERFORMANCE BOUNDS OF GAUSSIAN SCS 

Following [9], this section shows that with Gaussian and Bernoulli 
random matrices of &(k) measurements, considerably smaller than the 
& (k\og(N / k)) required by conventional CS, the average error of Gaus- 
sian SCS is tightly upper bounded by a constant times the &-best term 
approximation with overwhelming probability, the failure probability being 
orders of magnitude smaller than that of conventional CS. The proofs of the 
theorems will not be given due to the space limitations. We consider only 
the encoder-decoder pairs (<£,A) that preserve <£x, i.e., <£(A(<£x)) = <£x, 
satisfied by the optimal A in ([2} for Gaussian signals x, V4>. 



IV-A. From Null Space Property to Instance Optimality 

The instance optimality in expectation bounds the average error of SCS 
with a constant times that of the best £-term approximation (5j, defining 
the desired SCS performance: 

Definition 1. We say that A) is instance optimal in expectation of order 
k in || • \\x, with a constant Cq, if 

£^ jW ||x-A(te)|br <C o k ({x}) x , (7) 
the expectation with respect to x, and to <& if one random is drawn 
independently for each x. Similarly, the MSE instance optimality of order 
k is defined as 

£ Xj(<I)) ||x-A(*x)||2<CoCT,({x})2. (8) 

The null space property in expectation defined next will be shown 
equivalent to the instance optimality in expectation. 

Definition 2. We say that <E> in (4>,A) has the null space property in 
expectation of order k in || • \\x, with constant C, if 

VT]=x-A(<I>x), (9) 

the expectation with respect to x, and to <I> if one random <E> is drawn 
independently for each x. Note that r\ £ Null(<E>). Similarly, the MSE null 
space property of order k is defined as 

^(*)llnll2<C°it({i?})2> V77=x-A(*x). (10) 
Theorem 2. Given an M x N matrix <E>, a norm || • \\x, and a positive 
integer k<N, a sufficient condition that there exists a decoder A such that 
the instance optimality in expectation ^ holds with constant Cq, is that the 
null space property in expectation ^ holds with C = Co/2 /or this A). 
A necessary condition is the null space property in expectation {9} with 
C = Cq. Similar results hold between MSE instance optimality {SJ and null 
space property |T0| >, with the constant C = Cq/4 in the sufficient condition. 

Comparing to conventional CS that requires the null space property to 
hold with the best 2&-term approximation error (9|, the requirement for 
Gaussian SCS is relaxed to k, thanks to the linearity of the best &-term 
approximation for Gaussian signals. 

Theorem [2] proves the existence of the decoder A for which the 
instance optimality in expectation holds for (^>,A), given the null space 
property in expectation. However, it does not explain how such decoder is 
implemented. The following Corollary, a direct consequence of theorems [T] 
and [2] shows that for Gaussian signals the optimal decoder (|2| leads to the 
instance optimality in expectation. 

Corollary 2. For Gaussian signals x ~ ^/K(0, S), if an M x N sensing 
matrix <J> satisfies the null space property in expectation ^ of order k in || • 
||i, with constant Cq/2, or the MSE null space property |T0] > of order k with 
constant Cq/4, then the optimal and linear decoder A = S<E> r (<I>S<E> r ) -1 
satisfies the instance optimality in expectation (7} in || • ||i, or the MSE 
instance optimality (8}. 

IV-B. From RIP to Null Space Property 

The Restricted Isometry Property (RIP) of a matrix measures its ability to 
preserve distances, and is related to the null space property in conventional 
CS (7), 1 10]. The new linear RIP of order k restricts the requirement of 
conventional RIP of order k, to a union of ^-dimensional linear subspaces 
with consecutive supports: 

Definition 3. Let k < N be a positive integer. Let K\ define a linear 
subspace of functions with support in the first k indices in [l,iV], K2 a 
linear subspace of functions with support in the next k indices, and so on. 
The functions in the last linear subspace Kj defined this way may have 
support with less than k indices. An MxN matrix O is said to have linear 
RIP of order k with constant 8 if 

(l-5)||x|| 2 <||*x|| 2 <(l + 5)||x|| 2 , Vx€iy j=1 Kj. (11) 

The linear RIP is a special case of the block RIP fTTl . with block sparsity 
one and blocks having consecutive support of the same size. 



The following theorem relates the linear RIP (TT\ of a matrix to its 
null space property in expectation (9]). 

Theorem 3. Let x G M. n be a random vector that follows a certain 
distribution. Let <3> be an MxN matrix that satisfies the linear RIP of order 
2k with 8 < 1, and let A be a decoder. Let r\ = x — A(<E>x). Assume further 
that £ x ,(<&) I M I decays in n: £ x ,(<l>) I *7 [ w + 1] I < ^x,(<&) I *7 W l> Vn < iV. 77zew 
satisfies the null property in expectation of order k in \\ ■ \\\ ([9), with 
constant C = 1 + k 1 / 2 ^^ . 

For Gaussian signals x G ,/K(0,S), with Gaussian or Bernoulli 
matrices, one realization drawn independently for each x, and with A the 
optimal decoder ([2]), the decay of £ Xj <j>|77[n]| assumed in in Theorem [5] is 
verified through Monte Carlo simulations. 

IV-C. From Random Matrices to RIP 

The next Theorem shows that Gaussian and Bernoulli matrices satisfy 
the RIP for one subspace with overwhelming probability. 

Theorem 4. [2] Let <E> be a random matrix of size MxN drawn according 
to any distribution that satisfies the concentration inequality 



Pr(\\®x\\i - \\x\\i\ > £\\x\\i) < 2e- Mc ^°l l \ Vxe W\ (12) 

where < 8 < 1, and cq(8/2) > is a constant depending only on 8/2. 
Then for any set K C {1, . . . ,N} with \K\ = k < M, we have 

(l-<5)||x|| 2 < ||fe|| 2 < (l + «)||x|| 2 , Vxg4 (13) 
where -%k is the set of all vectors in R N that are zero outside of K, with 
probability greater than or equal to 1 —2(\2/8) k e~ c °^/^ M . Gaussian and 
Bernoulli matrices satisfy the concentration inequality fl2] ». 

The linear RIP of order k fTT] ) requires that |T5]l h olds for N/k<N sub- 
spaces. The next Theorem follows from Theorem]?] by simply multiplying 
by N the probability that the RIP fails to hold for one subspace. 

Theorem 5. Suppose that M, N and < 8 < 1 are given. Let <& be 
a random matrix of size MxN drawn according to any distribution 
that satisfies the concentration inequality (12) . Then there exist constants 
c\,C2 > depending only on 8 such that the linear RIP of order k {TT} 
holds with probability greater than or equal to 1 — 2Ne~ ClM for <3> with 
the prescribed 8 and k<c\M. 

Comparing with conventional CS, where the null space property requires 
that the RIP £5) holds for (JQ subspaces l2l. 171. flOl. the number of 
subspaces in the linear RIP (TT]) is sharply reduced to N/k for Gaussian 
SCS. In consequence, with the same number of measurements M, the 
probability that a Gaussian or Bernoulli matrix satisfies the linear RIP is 
substantially higher than that for the conventional RIP. Equivalently, given 
the same probability that <E> satisfies the linear RIP or the conventional 
RIP of order k, the required number of measurements for the linear RIP 
is M ~ G(k), substantially smaller than the M ~ @(k\og(N /k)) required 
for the conventional RIP. Similar improvements have been obtained with 
model-based CS that assumes structured sparsity on the signals 151 . 

With the results above, we have showed that for Gaussian signals, 
with sensing matrices satisfying the linear RIP |TT} of order 2k, for 
example Gaussian or Bernoulli matrices with 0(k) rows, with overwhelm- 
ing probability, and with the optimal decoder SCS leads to the 
instance optimality in expectation of order k in || • ||i (7]), with constant 
C = 2(1 + £ 1/2 {^f )• By the definition of CS, k 1 ! 1 is typically small. 

V. PERFORMANCE BOUNDS OF GAUSSIAN SCS 
WITH RIP IN EXPECTATION 

This section shows that with an RIP in expectation, a matrix isometry 
property more adapted to SCS, the Gaussian SCS MSE instance optimal- 
ity (8} of order k and constant Co, holds in the / 2 norm with probability 
one for any matrix. Co has a closed-form and can be easily computed 
numerically. 

Definition 4. Let x G be a random vector that follows a certain 
distribution. Let <E> be an M x N sensing matrix and let A be a decoder. 



Let r\ = x — A(<£x). <E> in (^*,A) is said to have RIP in expectation in K 
with constant ck if 

^,(*)ll^||2 = ^,(*)||T?r||2, VT7=x-A(<£x), (14) 
where K C {1,. . . ,N}, K]k £ ^ N is the signal r\ restricted to K (t\k\p\ — 
T][n], V n G K, and otherwise), and the expectation is with respect to x, 
and to <E> if one random <£> is drawn independently for each x. 

The conventional RIP is known to be satisfied only by some random ma- 
trices, Gaussian and Bernoulli matrices for example, with high probability. 
For a given matrix, checking the RIP property is however NP-hard 0. By 
contrast, the constant of the RIP in expectation (\A\ can be measured for 
any matrix via a fast simulation, the quick convergence guaranteed by the 
concentration of measure |16|. The next proposition, directly following 
from (5} and j4|, further shows that for Gaussian signals, the RIP in 
expectation has its constant in a closed form. 

Proposition 1. Assume x ~ c/K(0, S), is an MxN sensing matrix and A 
is the optimal decoder ([2). Then <£> in (<I>, A) satisfies the RIP in expectation 

= c K (E<p) [Tr (R/rSRj - R^SO r (OSO r ) _1 <I>SRj)] , (15) 
where R^ is anNxN extraction matrix giving r\x — Rs:^ i-e-> RfcO\ — h 
V/ G K, all the other entries being zero. The expectation with respect to 
is calculated if one random <E> is drawn independently for each x. 

The next Theorem shows that the RIP in expectation leads to the MSE 
null space property holding in equality. 

Theorem 6. Let x G M N be a random vector that follows a certain 
distribution, <& an MxN sensing matrix, and A a decoder. Assume 

Ex,0&) ll^lll ^ an d Ex,0$) W^kc ll 2 7^ 0> f or some K C {1, . . . ,N}. Assume 
that <E> in (^>,A) has the RIP in expectation in K with constant ax > 0, 
and in K c = {1, . . . ,N}\K with constant b^ > 0: 



E x ,^)\\^r] K \\l _ ^ x ,(d>)ll^%c|l2 

^x,(^))||%c||2 



Then <I> satisfies 



-a K , 



-bK, V T] = x AOx. 



E x,(<s>) II 7 ? II2 - CoE x ,(<t>) WIrc II25 



(16) 



(17) 



where Co = 1 + bg/ aK- In particular, if K = { 1 , . . . ,k}, then <E> satisfies the 
MSE null space property of order k, which holds with equality, 

E^Ml^CoGkii^l (18) 

Let us check the MSE null space property constant Co of different 
sensing matrices in SCS for Gaussian signals x G R N ~ o/K(0,S), assuming 
the eigenvalues of S follow a power decay (6} with a = — 3, and N = 64. 
Gaussian, Bernoulli and random subsampling matrices of size MxN 
are considered, and the optimal and linear decoder A ([2]> is applied to 
reconstruct the signals. For each matrix distribution, a different random 
matrix realization is applied to sense each signal x. Note that since the 
random subsampling matrix <£, each row containing one entry with value 
1 at a random position and otherwise, has the maximal coherence with 
the canonical basis, this matrix is not suitable for directly sensing x Q, 
and is replaced by in the simulation, with a DCT basis having low 
coherence with <£. 

Figure [5] (a) plots Co = 1 obtained by simulating |T6] >, with k = 

10, for different values of M. When the number M of SCS measurements 
increases, the reconstruction error of SCS decreases, resulting in a smaller 
ratio over the best-/: term approximation error with a fixed k. Gaussian and 
Bernoulli matrices lead to similar Co values, slightly smaller than that of 
random subsampling matrices. Figure [2] (b) plots Co, as a function of k, 
with M = k. Gaussian and Bernoulli matrices lead to similar Co ~ 4.5 that 
varies little with k. For random subsampling matrices Co slowly increases, 
almost linearly, and is equal to 5.5 for a typical value k = 10, about 20% 
larger than that of Gaussian and Bernoulli matrices. 

From Corollary [2] and Theorem [6] we obtain the next concluding result: 
Theorem 7. Assume x ~ c/K(0, S). Let be an MxN sensing matrix and 
A the optimal and linear decoder {2|. Then satisfies the MSE instance 
optimality of order k ([8} with constant Cq =4(1 ^b^/a^), clr an d b& 




— Gaussian 
^Bernoulli 
— Subsampling 




(a) (b) 
Fig. 2. The MSE null space property constant Co {18} of Gaussian, Bernoulli, 
and random subsampling matrices, as a function of M (left), and of k with 
M = k (right). 



given in ( |16| ). 



VI. SCS WITH GMM 
EXPERIMENTS ON REAL IMAGE DATA 



This section applies SCS with GMM in real image sensing and compares 
it with conventional CS based on sparse models. 

Following 1 18 1, an image is decomposed into \/N x y/N = 8x8 patches 
{xi}l<i<i assumed to follow a GMM: There exist K Gaussian distributions 
{^(H>k^k)}i<k<Ki an d each image patch x; is independently drawn from 
one of these Gaussians with an unknown index k. SCS samples each patch 
ji = fyxi, with a possibly different for each x;. The decoding scheme 
(i) estimates {(fJ<k^k)}l<k<K> an d (ii) identifies the Gaussian distribution 
ki that generates the z'-th patch, and reconstructs with the optimal and 
linear decoder {2} associated to the appropriate Gaussian, V/. The resulting 
piecewise linear decoder is implemented with a computationally efficient 
MAP-EM algorithm alternating between (i) and (ii) 1181 . The algorithm has 
fast convergence and the MAP probability of the measured data increases 
as the algorithm iterates (T8) (refer to fl~8l Sec. 2 for more details). 

The dictionary for conventional CS is learned with K-SVD (T] from 
720,000 image patches, extracted from the entire standard Berkeley seg- 
mentation database containing 300 natural images H5l . The decoder is 
calculated with an l\ minimization. 
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(a) (b) 
Fig. 3. (a). Patch PSNR (dB) vs sampling rate for SCS and CS using Gaussian 
and random subsampling sensing matrices, (b) Image PSNR (dB) vs sampling 
rate, for SCS using Gaussian sensing matrices with non-overlapping reconstruc- 
tion, and subsampling random matrices with overlapping reconstruction. 

Figure [3] (a) shows the sensing performance on about 260,000 (sliding) 
patches, regarded as signals, extracted from the standard image Lena, as 
shown in Figure [4] The PSNRs generated by SCS and CS using Gaussian 
and random subsampling sensing matrices, one independent realization for 
each patch, are plotted as a function of the sampling rate M/N. At the same 
sampling rate, SCS outperforms SC. The gain increases from about 0.5 dB 
at very low sampling rates (M/N ~ 0.1), learning GMM from the poor- 
quality measured data being more challenging, to more than 3.5 dB at high 
sampling rates (M/N ~ 0.5). (SC using an "oracle" dictionary learned from 
the full Lena itself, undoable in practice, improves its performance from 
0.2 dB at low sampling rates to 1.3 dB at high sample rates, still lower than 
SCS.) For both SCS and CS, Gaussian and random subsampling matrices 
lead to similar PSNRs at low sampling rates (M/N < 0.25), and at higher 
sampling rates the former gains about 0.5 dB. Recall that SCS is not just 
more accurate and significantly faster, but also only uses the compressed 
image, while CS uses a pre-learned dictionary from a large database. 

Figure [3] (b) shows the sensing performance on the whole Lena image. 
The sensing is performed on non-overlapped patches. Random subsampling 
matrices, which are diagonal operators (one non-zero entry per row), 
have the advantage of being able to lead to reconstruction on overlapped 
patches, averaging the overlapped reconstructed patches further improving 
the image estimation. The PSNRs generated by SCS using random sub- 
sampling matrices and overlapped reconstruction are plotted, in comparison 
with those obtained using Gaussian sensing matrices and non-overlapped 



reconstruction. The former improves from about 3.5 to 1.5 dB, at a cost 
of N = 64 times computation. As illustrated in Figure [4] the overlapped 
reconstruction removes the block artifacts and considerably improves the 
reconstructed image. 




Lena 



No.-ovl. rec. 30.54 dB Ovl. rec. 33.26 dB 



Zoom (original) 

Fig. 4. From left to right. Lena, a zoomed crop, reconstructed image by SCS 
using Gaussian sensing matrices and non-overlapping reconstruction, and by 
SCS using subsampling random matrices and overlapping reconstruction. The 
image is sensed on non-overlapped patches at a sampling rate of M/N = 0.25. 
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